We consider the problem of estimating the parameters of a $d$-dimensional rectified Gaussian distribution from i.i.d.

A popular generative model these days is as follows: pass a standard Gaussian noise through a neural network. But a major unanswered question is what is the structure of the resulting distribution? Given samples from such a distribution, can we learn the distribution parameters? This question is the topic of this paper. Specifically, consider a 1-layer ReLU neural network, which is specified by a matrix W and a real bias b.

We consider the problem of estimating the parameters of a d -dimensional rectified Gaussian distribution from i.i.d. A rectified Gaussian distribution is defined by passing a standard Gaussian distribution through a one-layer ReLU neural network. We give a simple algorithm to estimate the parameters (i.e., the weight matrix and bias vector of the ReLU neural network) up to an error \eps orm{W}_F using \widetilde{O}(1/\eps 2) samples and \widetilde{O}(d 2/\eps 2) time (log factors are ignored for simplicity). This implies that we can estimate the distribution up to \eps in total variation distance using \widetilde{O}(\kappa 2d 2/\eps 2) samples, where \kappa is the condition number of the covariance matrix. Our only assumption is that the bias vector is non-negative.

We consider the problem of estimating the parameters of a d -dimensional rectified Gaussian distribution from i.i.d. A rectified Gaussian distribution is defined by passing a standard Gaussian distribution through a one-layer ReLU neural network. We give a simple algorithm to estimate the parameters (i.e., the weight matrix and bias vector of the ReLU neural network) up to an error \eps orm{W}_F using \widetilde{O}(1/\eps 2) samples and \widetilde{O}(d 2/\eps 2) time (log factors are ignored for simplicity). This implies that we can estimate the distribution up to \eps in total variation distance using \widetilde{O}(\kappa 2d 2/\eps 2) samples, where \kappa is the condition number of the covariance matrix. Our only assumption is that the bias vector is non-negative.

We consider the problem of estimating the parameters of a $d$-dimensional rectified Gaussian distribution from i.i.d. A rectified Gaussian distribution is defined by passing a standard Gaussian distribution through a one-layer ReLU neural network. We give a simple algorithm to estimate the parameters (i.e., the weight matrix and bias vector of the ReLU neural network) up to an error $\eps orm{W}_F$ using $\widetilde{O}(1/\eps 2)$ samples and $\widetilde{O}(d 2/\eps 2)$ time (log factors are ignored for simplicity). This implies that we can estimate the distribution up to $\eps$ in total variation distance using $\widetilde{O}(\kappa 2d 2/\eps 2)$ samples, where $\kappa$ is the condition number of the covariance matrix. Our only assumption is that the bias vector is non-negative.